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(57) Abstract 

Methods and an apparatus arc described whereby Internet content is coordinated with audio communications, such that two or more 
parties can view the same media content on the Internet while also simultaneously communicating over a traditional telephony network or 
by using voice over network implementations. The visual interaction is achieved by having a user computer display shared content that 
correspond to a second computer's display, such that both parties will be viewing a copy containing the same content on each of their 
browsers. The visual interaction further provides a method for allowing either of the parties to update the visual content of their browsers. 
In a further embodiment the resource collection functions of the user computer are enhanced by having the server only download to the 
user computer a user applet containing the information required by the user computer as is necessary. In this way, the user computer docs 
not have to wait as long or dedicate resources to the visual interaction that arc not necessary at the time. In a further embodiment this 
method works through firewalls on the network. 
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Title: Method and Apparatus for Coordinating 
Internet Multi-Media Content with 
Telephone and Audio Communications 

Background of the Invention 

5 The present inveniion relates generally to an apparatus and method ibr 

coordinating Intemel communication between two users. More pariicularh . 
the inveniion relates lo an apparatus and method for coordinating Internet 
multi-media communication such that the two users can view and modifv a 
copy of the same mulu-media content, where any modifications made b\ one 
l(» user are subsequently viewed by a second user, as well as enhancinLr the 
communication by supplementing any visual communication with audio 
communication. 

Businesses commonly service customers through call centers staffed with 
technical support personnel, sales support personnel, or service personnel 
1 5 (hereinafter collectively referred to as sales representative) that can answer the 
customer's questions. Typically, these centers use an automated call 
distributor ("ACD") system to handle incoming calls from the telephone 
network. Commercially available ACD systems provide call processing, voice 
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processing, data processing and network capabilities for the call centers. The 
ACD system usually routes customer calls to a sales representative. The sales 
representative typically uses a headset and a user device connected to the ACD 
system to respond to the incoming calls. The ACD system can route the calls 
to queues, which can be customized to rellect how a business wants to 
prioritize and route incoming calls. 

While the customer is on-hold, the ACD system also provides the 
customer with information. typically, the ACD system will notify the 
customer that the customer's call is important, "please hold", suggest that the 
customer call back during non-peak hours, or inform the customer the time the 
customer can expect to be on-hold, for example. In such an implementation, 
the customer has to wait on the telephone while listening to the repeated 
messages of the ACD system. This can frustrate the customer. It also restricts 
the customer from perlbrming many other tasks while the customer is waiting 
on the phone for the sales representative or technical specialist. In some 
implementations, the ACD systems pipe advertising to the customer while the 
customer is on-hold. The customer, however, may not be interested in the 
products or services that are being advenised, and may have further frustration 
from being subjected to the advertisements. Alternatively, the ACD system 
can provides the customer with melodies during the on-hold period. In this 
implementation, however, the customer is still restricted from doing other tasks 
and the business looses an opportunity to advertise. 

Another way businesses communicate with customers is via Internet web 
pages. Businesses elicit questions from customers by advertising the 
businesses products on Internet web pages. Such web pages allow the customer 
requesting that a sales representative of the business contact them. 1 o request 
service, the customer inserts their name and e-mail address or phone number in 
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a space provided on the web page and activates a call-mc button. Thcreafler. 
the sales representative is sent an e-mail containing the request for service, lo 
which the sales representative responds. However, when the sales 
representative responds, either by phone or e-maih the customer is most likely 
5 no longer viewing the businesses web page, and thus the business's web page 
cannot contribute to the sales representative's pitch. Moreover, the response is 
not immediate, sometimes with log times measured in days, and during the 
interim the customer may buy another product. 

An intermediate solution to answer a customer's questions regarding 
U) products is presented by allowing the customer to use or view a demonstration 
of the business's product while on the Internet. By way of example, if a 
customer wants lo buy software, the customer may go to the businesses web 
site and request a software demonstration. The web site downloads to the 
customer the software demonstration. If all the customer's questions are 
15 answered by viewing the demonstration, then the customer can then purchase 
the software. If the customer still has unanswered questions, however, the 
customer will again have to either place a telephone call or wait for a delayed 
e-mail or call response. 

A further disadvantage of providing the customer a software 
0 demonstration downloaded from the Internet is the amount of time it takes a 
customer to go to each web site, ascertain the product the customer wants lo 
view, and then wait while the appropriate plug-in or code, for example, is 
downloaded to the customer's computer. If the customer only needs to view a 
part of the demonstration, downloading the entire code to a customer 
-5 unnecessarily uses resources of the customer. In addition, the time the 
customer waits for code to be downloaded is again wasted time, just as was 
waiting on the phone. One solution to this problem is taught in US Patent No. 
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5,572,643 to Judson which claims a method for displaying information on a 
graphic user interface when a customer is waiting for a link to download on the 
Internet. However, this is akin to the making the customer listen to unwanted 
advertisements on the phone and. thus, still has the previously described 
5 problems. 

A further disadvantage of downloading a .software demonstration is that 
whatever is downloaded is viewed by the customer alone without the assistance 
of the sales representative. The software demonstrations are not usually 
concurrent-use software applications that can be viewed simultaneously ai 
I) remote locations. Thus, the software applications cannot be viewed by both the 
customer and the sales representative simultaneously. Further, if they were, 
this would require even longer download periods. In addition, such 
applications would require a large amount of the cu.<;tomer\s resources to run 
and be difficult for the customer to manipulate. 
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Summary 

The aforementioned and other objects are achieved by the invention which 
provides an apparatus for coordinating Internet communication between at least 
two users and a method associated therewith. The apparatus provides a visual 
5 communication across a network between a user computer and a second 

computer which is in communication with a server disposed between the user 
computer and the second computer. 

The method facilitates visual communication across the network that is 
platform independent. As such, neither the user computer nor the second 
10 computer is restricted to using any particular operating system or software 
application. The only requirement of the user computer and the second 
computer is that it has a browser-like application for reading files. In the 
preferred embodiment, the browser is a hyper-text markup language C'l ITMl/*) 
browser. 

15 To provide the visual communication between the user computer and the 

second computer, a link is established between the user computer and the 
server and a link is established between the server and the second computer. 
The server, the user computer, and the second computer can be fully distributed 
across the network. 

20 In addition to being part of the link that is established, the server 

selectively transmits to the user computer a user applet. The user applet 
enables the user computer to communicate with the server. The second 
computer either has a service applet disposed thereon or the service applet is 
provided to the second computer by the server. The service applet and the user 

25 applet enable the visual communication to occur without requiring either the 

user computer or the second computer to have any executable files pre-installed 
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lo enable the user eomputer or the second computer to have a shared vieu . 
Kurther, the user applet and the server applet bi-directionally communicate whh 
the server, and thus can bi-directionaliy communicate with each other through 
the server. 

After the user computer has received the user applet, the user computer 
generates a user view. Either in parallel or thereafter, the server communicates 
that the link is established between the server and the user computer to the 
service applet. Thereafter, a session is selectively established between the user 
computer and the second computer. 



During the session, the second computer and user computer both 
display on the browser of their computer, respectively, a shared view in their 
user interfaces. The shared v.ew allows a user of the user computer to sec a 
copy of the same content as is displayed on the second computer, and vise 
versa. Thus, both users of the user computer and second computer will be 
I -■^ looking at the same content displayed on the browser. 

The user interface of the user computer is called a user view, while the 
user interlace of the second computer is called a service view. The service 
view contains a representation of the user view, the representation of the user 
view is the shared view. 

20 Either the user computer or the second computer can manipulate the 

shared view. For example, the second computer can change the shared view 
from content at one universal resource locator ("URL") source location to 
content at a second URL .source location. If the second computer implements 
change to the shared view, the change is communicated to the service applet. 

25 The service applet transmits the change to the server, and the server 

subsequently transmits the changes to the user applet. The user applet then 
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implements the change in the user view, and notifies the server thai ihc change 
has been implemented. After the service applet receives notification from the 
server that the user view has changed, the service applet changes the 
representation of the user view in the service view, thereby conforming the 
5 shared view of the second computer lo match the shared view of the user 
computer. This process enables the second computer to communicate to the 
user computer. The process can be performed bi-directionally, such that the 
user computer can change the user view and cause a conforming change in the 
second computer. 

10 Various implementations of this method can be enabled by the user 

computer or the second computer. One implementation allows changes to the 
URL source location of the shared view. Another implementation allows 
changes to the images in the shared view. A further implementation of this 
embodiment of the invention allows the shared view to be a software 

15 application. A still further embodiment allows the shared view to display 
software demonstration programs. 

An additional implementation of this embodiment is an interactive use oi' 
forms, where each form has objects. In this implementation, the form is 
represented as an array having arguments where the arguments represent the 
20 objects of the form. As the user computer or the second computer changes 

entries on the form, the user applet or the service applet, respectively, updates 
the objects in the array such that the array now represents any changes in the 
user interface. Then as previously described the changes are implemented in 
the shared view of both the user computer and the second computer. 

25 A further implementation for the interactive use of forms supplements the 

forms to be event monitoring, such that when the objects on the forms change 
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the objecis notify the applets of the change. A still further implementation 
rewrites the objects as applets. Again, the changes to the objects are 
implemented in the shared view. 

lo enable the visual communication, the server must download the user 
5 applet to the user computer. In one embodiment of the present mvention, the 
server enhances a resource collection function of the user s computer by only 
downloading to the user computer a user applet representative of the 
functionality the user computer needs at that time. As such, the server 
a.scertains what the user computer needs and generates an applet representative 

10 oi that functionality and downloads that applet to the user computer. If later on 
during the visual communication the user computer requires greater 
f unctionality, the server will then download a second user applet containing the 
necessary functionality. In this way. the server decreases the resource 
collection of the user computer in both time and space while enabling the 

15 functions of the resource that the user wishes to enable. 

The user applet is further enhanced by the use of a script. 1 he script is 
displayed on the user interface of the second computer, preferably adjaceni to 
the representation of the user view . The display can be a list of scripts irom 
which the second computer can choose or the script that the user of the second 
20 computer has selected. The script is in communication with the service applet. 
The script has simplified commands which it transmits to the service applet lo 
enable the service applet to implement any changes in shared view and transmit 
the changes to the user applet as previously described. In the preferred 
embodiment the script is written in hyper-text markup language (^'HTML'*). 

25 The visual communication can be between more than one user computer 

and more than one second computer. This enables many-to-one, one-to-many. 
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or many-io-many communication between al Icasi one user computer and at 
least one second computer. Each of the more than one second computer can be 
identical or each of the more than one second computer can have differing 
service applets which are adapted to perform differing functions, such as 
5 administrative, sales, or technical support. 

If the embodiment has more than one second computer, the server also has 
more than one queue to which it routes call requests from the user computers. 
A queue contains one or more call requests from the user computers. *I'he 
queue is further displayed selectively on one or more second computers. The 

10 server routes call requests using various methods, such as a lirsi-come. first- 
serve basis, according to attributes of the user computers, sales specialty of the 
second computers, or logic formulas, which may comprise formulas relate to 
time of call, number of call requests presently in queue, among other items. 
The attributes can be collected by prompting to the user computer, or the user 

15 applet can extract the attributes from the user computer. 

When the call request of the user computer is placed in a queue, the 
server notifies the user applet regarding status of the of the queue user 
computer's call request. The server will display to the user computer in its user 
interface the status of the queue, enabling the user computer to know when his 
20 call request will likely be answered. 

In one embodiment, while an entry of the user computer is in the queue, 
the server through the user applet displays supplemental information which is 
analogous to MUZAK, a trademark of Muzak, 1 nc. on the user computer until 
the user computer's call request is answered. In one embodiment, the 
25 computer supplemental information is selectively transmitted from the server 
based upon the attributes of the user computer. In another embodiment, the 
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server transmits the computer supplemental information based upon scripts. 
While the user computer is being displayed the computer supplemental 
inrormation. the user computer also has a control view generated by the user 
applet that enables the user computer options such as disconnect or call me 
5 later, among other options. 

In still another embodiment, the visual communication across the 
network is married to audio communication between the user computer and the 
second computer. In one implementation of this embodiment, both the 
computer and the second computer have a voice-over net capability that 

10 enabled durmg the session between user computer and second computer. In 
another embodiment, a traditional telephony system is used to implement the 
audio communication during the session. In this embodiment one audio device 
will be associated with at least one .second computer and at least one user 
computer. The audio device can either initialed the communication or be u.sed 

1 5 after the initiation of the visual communication between the user computer and 
the second computer. As such, the users of the user computer and the second 
computer can view shared content over the network as well as audibly 
communicate. 



20 



25 



user 
is 



In the implementation where the audio communication is established first, 
the second computer directs the user computer to an appropriate address, or 
URL source location, whereupon the user computer is linked to the server and 
provides a user applet as previously described. In this embodiment, the user 
computer can also selectively be provided a password to enter the call session, 
thus enabling the call session to be private. 

In still another embodiment, the second computer can be a component of a 
call center where the call center has an automated call distribution svstem 
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("ACD"). In this embodiment, the ACD also has a call queue. The call queue 
can be communicated to the server via the network if the ACD is also 
connected to the network, and as such the call queue of the ACD can work in 
tandem with or complement queue of the server. In this embodiment, the call 
5 center can utilize any additional capabilities of the ACD while supplementing 
the ACD with visual communication enabled by the server, the user applet, 
and the service applet. 

In another embodiment, the communication between the server and the 
user computer can be conducted through a firewall using tunneling. 

10 In further aspects, the invention provides methods in accord with the 

apparatus described above. The aforementioned and other aspects of the 
invention are evident in the drawings and in the description thai ibilows. 
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Brief Description of the Drawings 

The foregoing and other objects of this invention, the various features 
thereof, as well as the invention itself, may be more fully understood from the 
following description, when read together with the accompanying drawings in 
5 which: 

Figure 1 shows a system diagram of one embodiment of this invention; 
Figure 2 shows a block diagram of the system of Figure 1 ; 

Figure 3 shows a user interface m accordance with one cmbodmieni of this 
invention; 

Figure 4A shows a second user interface in accordance with one 
embodiment of this invention; 

Figure 4B shows a third user interface of another embodiment of this 
invention; 



Figure 5 shows a system diagram having more than one second computer 
1 5 in accordance with another embodiment oi' this invention; 

Figure 6 shows a system diagram of yet another embodiment t)f this 
invention; 

Figure 7 A shows a How chart of one implementation of how a user 
computer enters a call session in accordance with this invention; and 

20 Figure 7B is a flow chart of a second implementation of how a user 

computer enters a call session in accordance with an embodiment of this 
invention. 
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Detailed Description 

While the present invention retains utility within a variety of networks and 
may be embodied in several different forms, it is advantageously employed in 
connection with the Internet. Although this is the form of the preferred 
5 embodiment and will be described as such, this embodiment should be 
considered illustrated and not restricted. 

Figure 1 shows a system 10 according to one embodmicnt of the present 
invention. As shown, the system 10 includes a user computer 12 in 
communication with a network 16 and a second computer 24 in 
10 communication with the network 16. The system 10 further comprises a server 
20 interposed between the user computer 1 2 and the second computer 24 also 
in communication with the network 16. 

In the preferred embodiment, network 16 is the Internet. As such, 
communication there over is via Transfer Control Protocol/Internet Protocol 
15 C'TCPMP"). It should be obvious to those skilled in the art. however, that 
network 16 could be an intranet or any network capable of supporting 
communication protocols. 

Similarly, the system 1 0 can be partially distributed, as opposed to the 
fully distributed depiction of Figure 1 . For instance, the server 20 and the 
20 second computer 24 could be connected to a local area network ("LAN") or a 
wide area network (''WAN") which, in turn, is connected to network 16. 

The user computer 24 and the second computer 24 can be a personal 
computer, workstation, terminal, laptop or any device capable of displaying a 
user interface and storing an applet. The user computer 12 and the second 
25 computer 24, however, must have a browser 18, 28. respectively, where a 
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browser shall be defined herein as any application software or firmware that 
can interpret files to create a view thereon, the files include for example. 
HTML, Javascript. An example of such browser includes, for example, any 
common Internet browser, inter alia, and a Java workstation, inter alia. As 
described hereinafter m more detail with reference to Figures 7A and 7B, the 
user computer 12 and the second computer 24 access the server 20 through the 
browsers 1 8. 28, respectively. The browsers 1 8, 28 are displayed on the user 
computer 12 and the second computer 24 via a display 14, 26, which is a 
cathode ray tube or a liquid crystal display, for example. 

Once the user computer 1 2 accesses the server 20 utilizing the browser 1 8. 
the server 20 transmits to the user computer 12 a user applet 22. The user 
applet 22 is adapted to enable the user computer 12 to communicate through 
the network 16 with the server 20. A service applet 30 is disposed on the 
second computer 24 to enable the second computer 24 to communicate with the 
server 20. The service applet 30 can either be transmitted to the second 
computer 24 by the server 20 or be disposed thereon. After the user computer 
12 receives the user applet 22, the second computer 24 is notified that a link is 
established, and thai a session can be established between the user computer 12 
and the second computer 24. When the session is established, users of the user 
computer 12 and the second computer 24 can visually communicate. 

The users of the user computer 12 and the second computer 24 visually 
communicate by sharing content on their respective displays 14. 26. More 
panicularly, the browser 1 8 displays a user interface containing a user view on 
the user computer 12, while the browser 28 displays a user interlace on the 
second computer 24. When a session is enabled, the user interface of the 
second computer 24 contains a representation of the user view that is displayed 
on the user computer 12. Thus during a session, the user computer 12 and the 
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second computer 24 arc viewing a copy of ihc same user view on their 
respective browsers 18, 28. The user computer 12 and the second computer 24 
are then viewing shared content. 

During the session, either party can manipulate the shared content 
5 displayed on the party's computer. When the shared content is manipulated, 
the applet disposed on that party's computer will communicate to the server 20 
through the network 16 that there has been a change to the shared content. In 
the preferred embodiment, the applets 22, 30 selectively poll the server 20 
during a session to ascertain whether an event has occurred, such as a change in 

10 the shared content. This is facilitated by designating each session and each 
applet in the session with a unique identifier, as further described with 
reference to Figures 7A and 7B. When an applet polls the server 20 having the 
proper session identifier, the server 20 communicates to the applet that an event 
has occurred. That applet then communicates with the browser of its respective 

15 computer and causes the browser to display the any changes in the shared 
content. Once the browser of the unchanged computer displays what was 
communicated to it by the server 20. both parties will again be looking at the 
same shared content on their respective browsers. This process continues back 
and forth such that either party may lead the other to view, for instance, another 

20 web page or a software demonstration, among other items hereinafter 
discussed. 

In another implementation, the server 20, instead of the applet, can retrieve 
the changes to the shared content and then transmit the changes to the applets 
in the session. In this implementation, the server 20 acts as a proxy server, the 
25 function of a proxy server is well known in the art. 
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In the preferred embodiment, after a user of the seeond computer 24 elects 
to alter the shared content, the service applet 30 waits for a response from the 
server 20 before updating the shared content on the second computer. It wails 
for a response from the server 20 that it has communicated the user applet 22 
and has changed the shared content on the user computer 12. Once the server 
20 receives communication back from the user applet 22 that it has changed the 
shared content on the browser of the user computer 12, the server notifies the 
service applet 30 of the second computer 24. At this time, service applet 22 of 
the second computer 24 updates the representation of the shared content 52 ol' 
the second computer's interface. In this manner, the user interface of the 
second computer 24 will not gel ahead of the user computer 12. Ii should also 
be noted that each computer 12. 24 are retrieving the shared content 52 
independently and, in this instance, not sending the shared content 52 Irom one 
computer to another. 

5 It should be obvious to those skilled in the an that the server 20 can 

mform the applets 22, 30 in each session of any changes without requiring the 
applets 22. 30 to poll the server 20 without departing from the scope of this 
invention. Further, the applets 22, 30 can work in conjunction with the server 
30 such that any combination of polls or pushes results in the applets 22, 30 

0 being notified that an event has occurred. Moreover, polling can be replaced 
with event driven technology, or similar notification technologies, as the event 
driven technology becomes more acceptable on the Internet. 

In the preferred embodiment, the server 20 is a JAVA server utilizing 
JAVA objects. JAVA is a registered trademark of Sun Microsystems. Inc. fhe 
5 user applet 22 and the service applet 30 communicate with the objects disposed 
upon the server 20. In the preferred embodiment, the applets 34 are also JAVA 
applets. This enables the system 10 to be a cross-platform system and as such 
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does noi need any executable files previously disposed on the user compuicr 12 
or any of ihe computers involved in the session. The JAVA server can run on 
a UNIX computer, a MACINTOSH computer, MS Windows computer, a 
Silicon Graphics computer, or any other well-known computers in the art. 
5 UNIX is a registered trademark of AT&T Corp. MACINTOSH is a registered 
trademark of Apple, Inc. 

The user applet 22 performs event monitoring and communication as was 
described in part above. The user applet 22 also has a model controller view. 
The model controller view of the user applet 22 is the user view which is 

10 further described with reference to Figures 4A and 4B. The user applet 22 also 
shares much of the object model of the system 10, unlike a typical client. The 
user applet 22 having a portion of the object model allows the system 10 to 
function when it is fully distributed. This implementation makes it easy to 
replace the polling mechanism with distributed object communication 

15 technologies including, for example. Remote Method Invocation ("RMI") © 
Javascript. Sun, Common Object Request Broker Architecture ("COBI^'"). 
Internet Inier-Orb Protocol CTIOP'l and Distributed Component Object Model 

DCOM''), © Microsoft. For example, the system 10 can manipulate an 
object on the user computer 12 and have the user computer 12 transmit through 

20 the user applet 22 to the server 20 a response, both while the user computer 1 2 
is unaware that the communication or manipulation of the object has occurred. 
This implementation, among other things, enables the server 20 to extract from 
the user computer 12 attributes of the user computer 12 where the attributes can 
include items such as the user computer's e-mail address, the user computer's 

25 history of links on the network 16. and the user computer's specifications, 
among other items. 
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In the preferred embodiment, the user applet 22 is a persistent applet. The 
user applet 22 is loaded into a cache (not shown) of the user computer 12, 
preferably a local cache. It is persistent because once the user applet 22 is 
downloaded from the server 20, the user applet 22 remains on the user 
computer 1 2 as long as the user computer 1 2 remains in the session with the 
server 20, unless it gets removed from the local cache by the user computer 12 
as is well known in the art. However, if it is not removed, the persistent applet 
will remain disposed on the tiscr computer 12 such that it will not have to be 
downloaded again from the server 20. The user applet 22 will remain in the 
cache regardless of any changes to the user interface of the user computer 12. 

The user applet 22 is selectively generated by the server 20 to enhance a 
resource collection function of the user computer 12 while enabling the user 
computer 12 to participate in the visual communication. To enhance the 
resource collection function of the user computer 12, the server 20 evaluates 
the functions of a resource, such as a spreadsheet application for example, that 
the user computer 12 requires at any given point in time. After the server 20 
evaluates these functions, it then generates the user applet 22 representative of 
the functions. Once the server 20 generates the user applet 22 representative of 
these functions, it transmits the user applet 22 to the user computer 1 2. as 
previously described. 

Typically, the functions represented in the user applet 22 are only a subset 
of the functionality of the resource. This is analogous to using EXCEL, a 
trademark of Microsoft Corp., spreadsheet and only getting the graphing 
feature when one tries to use it, for example. By only including a subset of the 
functionality as required by the user computer 12, the time for downloading 
between the server 20 and the user computer 12 is decreased. Further, the 
amount of storage space required in the user computer 12 is also decreased. 
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Thus, the server 20 by selectively generating the user applet 22 enhances the 
resource collection function of the user computer 12. It should be obvious to 
those skilled in the art that the server 20 can, in real-time or pursuant to a timer, 
continually download to the user computer 1 2 any necessary functionality in an 
5 applet 22. The user computer 12 can also trigger the server 20 to inform it that 
it needs to be given an additional applet. In the system 10, the user computer 
1 2 receives what it needs as it needs it. Thus, the user computer 12 is 
optimized as a very thin client in the thin client fat server model. 

For example, if a user computer 12 enters a session to share a web page as 
10 further described hereinafter, the server 20 only downloads functionality to the 
user computer 12 necessary for the browser 1 8 to be notified about, and to 
display the web page. If for example, the user computer 12 is sharing a 
software application, then the server 20 will download to the user computer 12 
another applet in real time of the next piece of functionality the user computer 
15 12 needs to implement the software application. 

It should be obvious to those skilled in the art that the selective generation 
of an applet and subsequent downloading of that applet can be used for 
applicatiohs outside of the system 10 without departing from the scope of this 
invention. 

20 Turning now to the service applet 30, the service applet 30 can comprise 

one or more applets each adapted to provide the second computer 24 with 
different functionality. The service applet 30 at minimum must provide the 
second computer 24 with the ability to communicate with the server 20. 
Again, the server 20 can download the service applet 30 in real time to the 

25 second computer 24 during the session. The service applet 30 shares much of 
the object model with the user applet 22, as previously described, to enable 



wo 97/42728 



-20- 



PCT/US97/0r;40 



10 



implementation of the system 10 in a distributed environment. One 
implementation uses a proprietary interface definition language to 
communicate session inlbrmaiion between applets 22. 30 and the server 20 
based on a polling model, while another is based on event notification. Thus, 
the service applet 30 and the user applet 22 enable the user computer 12 and 
the second computer 24 to bi-directionally communicate over the network 1 6. 

The visual communication of system 1 0 is further enhanced by audio 
communication between the users of the user computer 12 and the second 
computer 24. In one implementation, the user computer 12 connects to the 
server 20, receives the user applet 22. and the service applet 30 is notified of a 
call request from the user computer 12, Upon notification of the request, the 
second computer 24 implements audio communication cither through voice 
over network ("VON") applications or a traditional telephony system (not 
shown). .At this point, the user computer 12 and the second computer 24 are in 
1 5 communication both audibly and visually. 

The visual communication can be further described with reference to 
Figure 2, which shows a block diagram of the system of Figure 1 . As was 
previously described, the server 20 transmits through the network 16 applets 
34. 1 he applets include the user applet 22, the service applet 30. any other 
20 applet necessary to give the user computer 1 2 or the second computer 24 
funciionaliiv. 



The server 20 is capable of ascertaining, as hereinafter described, what 
type of computer is requesting a download of one of the applets 34. This is 
achieved by having participants, such as the user computer and the second 
computer, in a session use different sign-on pages and/or by analyzing an 
information payload sent by the browsers in an HTTP sign-on request. As 
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such, if the server 20 identifies that the requester is the user computer 12 then il 
downloads the user applet 22 capable of generating a user view 90 and the 
functionality the user computer 12 requires as previously described. If. on the 
other hand, the server 20 ascertains that the requester is a sales representative. 
5 then it downloads a service applet 30 capable of generating a sales view 50. 
Similarly, if the server 20 ascertains that the requester is a system 
administrator, the server 20 downloads an applet capable of generating an 
administrator view 36. 

The server 20 ascertains and validates which type of user is requesting one 
10 of the applets 34 by various methods. One such method allows the server 20 to 
validate each of the computers using a password system. In this 
implementation, any requester can get any view 36, 50, 90 as long as it has the 
appropriate validation codes such as a name and a password. Additionally, 
with the appropriate validation codes a requester can log-on from anywhere in 
15 the world that is connected to the network 1 6. Thus, others can join in a 
current session simply by knowing the validation codes. 

In the preferred embodiment, the sales representative and the administrator 
have predetermined passwords stored in a database (not shown) 

20 communicating with the server 20. The database can be disposed on the server 
20 or remotely therefrom. The server 20 communicates with the database 
through a database interface that uses Java Database Connectivity {'"JOBC ) or 
other Object Database Connectivity C'ODBC") equivalent database languages, 
for example. In the preferred embodiment the database is a relational database. 

25 although it can be an object database or other forms of data storage well known 
in the art. 
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As such, when a requester log-on to the server 20, the server 20 checks the 
database to see whether the requester has a valid log-on and the associated 
applets 34 required by the requester. Once the server 20 validates the log-on, 
then as described, the server 20 will download one of the applets 34. In this 
way, the administrator can function as a sales representative havnig at one 
lime a service applet 30 downloaded to it via the server 20 and at another lime 
have an administrator applet downloaded to it via the server 20. 

The user computer 12 communicates with the server 20 through a web 
page. The user computer 12 either requests a call through the web page or 
enters a password on the web page that was provided to the user computer by 
the sales representative, as described further with reference to Figures 7A and 
7B. J bus, the server 20 can uniquely identify that a user computer 12 is 
requesting a user applet to generate the user view 90 by the method of 
communication with the server 20. 

Each of the applets 34 allows the browser of the respective computer to 
generate the associated views 36, 50, 90. In a session, the shared content is 
displayed in the user view 90 of the user computer 12 by the browser 1 8. while 
the shared content is displayed on the second computer 24 within the sales 
view 50. The user interface of the second computer 24 or the sales view 50 
contains a representation of the user view 90. 

When the shared content in the user view 90 or the representation of the 
user view is manipulated by one party, through communication among the 
applets 34 and the server 20 the shared content of the other party s view will be 
updated. For example, suppose the sales view 50 contains an HTMI. file 
located at the URL source location www.webline.com. The user view 90 will 
also contain a copy of the HTML file with the URL source location 
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wwvv.weblinc.com. If ihe second computer 24 wishes to direct the user 
computer 12 to the service's page of the Webline Corporation located ai 
www.webline_scrvices.com, then the second computer 24 can change its sales 
view 90 such that the URL source location of the shared view is 
5 www. webline_services.com. Once the second computer 24 changes the sales 
view 50 by methods such as the script as described hereinafter, or just by 
typing in the URL source location, the URL source location of that page is then 
transmitted by the service applet 30 to the server 20. The server 20 then tells 
every other applet that is part of the session that there is a new URL source 

K) location. In this example, the server 20 will tell the user applet 22 that there is 
a new URL source location. After the user applet 22 receives information, it 
then causes the browser 1 8 of the user computer 1 2 to pull down a new page 
from the network 16 and display on the user view 90 a copy of the HI MI. file 
at www.webline^services-com. The shared content can include web pages. 

15 files, application images, advertisements, interactive forms data, or application 
data among any other form of data that can be captured and displayed on the 
browser. 

In addition to the sales view 50 and user view 90, other views can also be 
implemented by the applet such as an administrative view 36. for example. 

20 The administrative view 36 displays information such as the structure of the 
sales force, reports, real-time monitoring of the communications, queue 
structure and status, which will be further described hereinafter, for example. 
It should obvious to one skilled in the art that the administration view 36. the 
sales view 50, and the customer view 90 can be disposed on one computer or 

25 more than one computer. 

Further any computer displaying the views 50, 90, 36 require a particular 
operating system 38 to accept the applet and share content in the preferred 
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embodiment. As such, they can be Windows-based. Macintosh-based, or 
UNIX-based, among others. Further, the browsers 40 can be any number of 
types, such as Netscape Navigator, Microsoft IE, Sun Hot Java, for example. 
Further, the browsers 1 8, 28 do not have to be the same on the user computer 
12 and the second computer 24. 

Turning now to Figure 3, which shows at least one user interface that in 
the preferred embodiment is the sales view 50. hi addition to a representation 
of the user view 52 thai contains the shared content, the user interface also 
contains a script screen 54. 

1 he script screen 54 can display a plurality of scripts 58 or just one of the 
plurality scripts that is activated in the user interface. The second computer 24 
can have predetermined scripts 58 illustrated on script screen 54 from which to 
choose, where each of the scripts 58 may be tailored to assist a particular type 
of user computer 12 or second computer 24. For example the scripts 58 could 
include: information regarding a company, a state of the market script, features 
of products, benefits, architecture or any set of information limited only bv the 
imagination of the user. In the preferred embodiment, the scripts 58 are written 
in HTML so that the scripts 58 are displayed in the browser. 

The scripts 58 dictate the shared content contained in the representation of 
the user view 52. The scripts 58 perform this function by having links that 
push the shared content into the representation of the user view 52. which, as 
previously described, is also pushed into the user view of the user computer 12. 
The scripts 58 perform this function by communicatmg with the service applet 
30. As the service applet in the preferred embodiment is a Java applet, the 
script communicates with the service applet 30 using Javascript-to-Java 
communication in which a Javascript command can invoke a public method in 
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a Java applet. It should be obvious to one skilled in the art that other 
embodiments are possible. Similarly, in another implementation the scripts 58 
can communicate directly with the server 20, instead of the service applet 30. 

The scripts 58 control what is displayed in a section of the browser 
5 adjacent to the scripts. As such, the scripts 58 update the browsers in a manner 
different than that is currently done for HTML files. 

An example of a script in the preferred embodiment is as follows: 

<html> 
10 <head> 

<title>Web.Dialog Sales Screen</title> 

</head> 
<body> 

<font size=5 color=brown><b> Web. Dialog Sales Screen </b> </font> 
15 <br> 

<font sizc=2> <b> Web.Dialog version 1 .0 Beta 1 </b> </font> 
<hr> 

<foni size=4> 
<ul> 
20 <li><a 

href= ''javascript rtop.salesrepFrame.documenl.SalesRepApplet.sctAddress 
<'http://www.webline.com") ''>Share ww\^'. webline.com </a></li> 
<br><br> 
<li><a 

25 href= * javascript: top. salesrepFrame. document. SalesRepApplet. set Address 

("http://www.microsoft.com') *'>Share www.microsoft.com </a></li. 
<br><br> 
<li><a 

href= ''javascript:top,salesrepHrame.document.SalesRepApplel.setAddrcss 
30 ('hitp://v\rv\^.netscape.com') ''>Share www.netscape.com </a></li> 
</ul> 
</font> 
</body> 
</html> 

35 In this example, the script is an HTN4L .lava script, which contains 

Javascript to communicate with the Java applet. The scripl is displayed in the 
top sales representative frame of the user interface, where the \op sales 
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representative frame is the script frame 54. In the code 

hrcf=Javascript:lop.salesrepFrame.document.SalesRepApplet.setAddress, 

set Address is a public method in the Java applet, which takes as an argument a 

URL source location, that instructs the applet to share the URL source location 

5 between all the participants in the session, being the user computer 12 and the 
second computer 24 in one embodiment. This particular script, if enabled by 
the second computer 24. will allow the second computer 24 push into the 
shared content the HTML file having a URL source location 
htlp:/Avv^-. wcbline.com, http://wvvu^microso^l.com or 

0 http://www.neiscapexom. 

For example, if the user of the second computer 24 wants to push 
htlp://wv/w. webhne.com into the shared content. In this example, the service 
applet 30 will receive from the script that the second computer 24 wants the 
shared content to display the file having the URL source location 
http://www.wcbline.com. The service applet 30 will then waits for a response 
from the server 20 that the server has verified that the user applet 22 has 
instructed the browser of the user computer 12 to download from the network 
16 the file having the URL source location http.V/wrww.weblincxom. Once the 
service applet 30 is notified that the server 20 has received the address of the 
shared content, the service applet 30 of the second computer 24 updates the 
representation of the shared content 52 of the second computer's interface, and 
the user applet 22 would receive the address of the shared content and update 
the shared content. The process would be repealed if the second computer 24 
then selected to push the www.microsoft.com file into the shared content. 

It should be obvious to those skilled in the art that scripts 58 can be used to 
push all the forms of shared content herein discussed lo the user computer. 
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including, scripts 58 lhal initiate a file transfer lo download information lo the 
user view from a remote location, for example. 

As the scripts 58 are in HTML, users can easily write them. Scripts 58 can 
even be written interactively by allowing the user of the second computer 24 to 
5 insert the arguments from the script by simply going from one page to the next 
while having the script translate URL of that web page and insert it as an 
argument in its script. The scripts 58 can also be dynamically defined to push 
shared content according lo attributes of the user computer 12 and/or attributes 
of the second computer 24. As such, the scripts 58 can be dynamically defined 
10 by the server 20 that access the attributes. 

in addition, the second computer 24 can dynamically generate scripts 
during the session to meet any requirements of the session. For example, the 
second computer 24 can create a script to push the product comparison into the 
shared content in response to a request of the user computer 1 2 lo see a 

15 product comparison. For instance, the second computer 24 can search for 
information regarding products, by keywords. URL source locations, for 
example. After selecting the information the second computer 24 wants to 
push into the shared content and thus the content of the script^ the second 
computer 24 can activate the script so generated. This selection can be 

20 implemented by default or by allowing the second computer to click a generate 
script button or an accept button displayed in a session box 62. for example. 
Again, this would allow the user computer 12 to view the product comparison. 
Any comparison information can be displayed lo the user computer 12 in ihis 
manner. Similarly, presentations regarding other information can be conducted 

25 by dynamically generating scripts and then pushing the shared content to the 
user computer 12. 
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Adjacent to the script screen 54 and disposed below it is a user information 
screen 56. The user information screen 56 contains queue information 60 and a 
session box 62. The session box 62 contains information about the session in 
which the second computer 24 is engaged in currently. I'he session information 
can include items, for example, duration of the session, address currenily being 
displayed in the representation of the shared content 52, any password 
associated with the session, status of the session, ACD interaction information, 
customer information and control functions such as connect 64, hang-up 66, 
pause, new view, log-off, help, among others. 

The session box 62 augments the scripts 58 by also providing a URL box 
67. Instead of using a script link, the second computer 24 can insert any URL 
into the URL box 67, enable the connect 66 button, and push the shared content 
to the user computer as previously described. 

The second computer 24 is also part of a queue of the server 20, and as 
such status of the queue information 60 is displayed on the user interface 50. 
The system 10 has at least one queue, where each queue contains one or more 
call requests from user computers. In the simplest implementation, when a 
user computer 12 links to the server 20, the ,server 20 places a call request from 
the user computer 12 in the queue in the order the server 20 received the 
request. 

I he call request can also be routed by the server 20 into queues based 
upon attributes of the user computer 12, where attributes of the user computer 
1 2 can be an e-mail address, HTML pages that the user computer 1 2 was 
viewing when it made the call request or interests, or database entries for a 
user, for example. The server 20 selectively places a call request of the user 
computer 1 2 by parsing the attributes into one or more of the queues. For 
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example, if the user attribute is thai the last page the user computer 12 viewed 
advertised laptops, the server could route the call request to a queue for laptop 
computers. The service applet 30 would then poll the server 20. collect the 
queue information and thereafter display it on the user interface 50 if the 
5 second computer 24 was assigned to the laptop queue. 

The assignments of the second computers are stored on the database 
disposed on the server 20, as well as information regarding the system, such as 
how long does it usually take for each call to be answered, for example. The 
database also contains information collected by the server 20 such as how 
10 many second computers are currently logged-on. how many second computers 
are in each queue, how many call requests arc already in each queue, whether 
any second computers 24 are on breaks and how long or short breaks, and 
whether more than one second computer 24 should be signed off. 

In one embodiment, the server 20 utilizes the database information and 
1 5 logic rules in association with the attributes of the user computer to route the 
calls lo the queues. For instance, the server 20 could route a user computer to a 
secondary laptop queue, perhaps one with less experienced sales 
representatives, when the primary laptop queue had a predetermined number ol' 
call requests. In this example the server 20 could also download an 
20 administrative applet to the administrative view to update the queues 

information 60 displayed on that view, or the server could cause a beeper to 
ring, for example. The server 20 can also compare the attributes, such as name, 
against a database of customers, to enable priority customers to be routed 
accordingly. 

25 The attributes could be collected from the user computer 12 using various 

methods. First, the user computer 12 could be prompted to enter the 
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mformation directly, or by choosing from a drop down list of predefined 
attributes, such as novice user, experienced user, for example. In still another 
embod.meni, the attributes can be extracted from the user computer 12 

The service applet 30 retrieves the queue information lo display usmg an 
interface language that includes commands, for example, such as, for example, 
tell mc how many other are call requests are in the queue. It should be obvious 
to those skilled in the serv.ce applet 30 can implemem polling or distributed 
object communication technologies without departing from the scope of this 
invention. The service applet 30 generates the queue information box 60. 
unlike the user applet 22. 

A user computer 12 receives status regarding lis position in the queue 
througli the server 20, a.s well as receivmg supplemental information, where the 
supplemental information can be analogized to MUSAK, a trademark of 
Musak, Inc. The supplemental information can be general information or 
specialized information displayed for the user computer 12. The server 20 
ascertains which specialized mformation to display to the user computer 12 
using the attribute information of the user computer 12. For example, if the 
attribute of the user computer 12 is laptops, the supplemental information 
displayed to the u.ser computer 12 while in the queue can be advertisements, 
accessories or specifications laptops, for example, among other information 
that may be of interest to a u.scr who wants to purchase a laptop. The server 20 
implements displaying the supplemental information by communicating with 
the user applet 22. 

Af^er the session is established and the user is out of the queue, the user 
view 90 will display the shared coment 52. In addition to the sharing a file as 
previously described, the user computer 12 and the second computer 24 can 
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share a demonstralion. To share a demonstration, either the user computer 1 2 
or the second computer 24 must have the application capable of running the 
demonstration on the computer, in the preferred embodimenl the second 
computer 24 has the apphcalion while the user computer 12 only has a 
5 browser. As the software demonstration runs, the second computer 24 captures 
an image of the demonstration, like a prim screen, and transmits the captured 
image to the user applet 22. The captured image is saved as a jpeg, gif, bmp 
flic or other graphical utility, transmitted to the server 20 or the user computer 
12 and displayed on the user computer J 2. 

U) The user computer 12 and the second computer 24 can also share a form. 

The form has a URL source location 70 and a place to enter user information 
74 and some text 78. In the preferred embodiment, the form can be any HTML 
page. The service applet 30 parses the form and stores each of the entries in 
the form as objects, such that the URL source location 70 will be an object, the 
text 78 would be an object and user information 74 would be an object, for 
example. In one iinplementaiion. the service applet 30 stores the objects as an 
array. In this example, it would be a two dimensional arra\ of three objects. 
Either the objects that have been changed can be transmitted by service applet 
30 to the server 20. or the array containing all the objects can be transmitted by 

0 the service applet 30 to the server 20, whereupon the user applet 22 would 
parse the girray to ascertain which objects had changed. 

If the sales representative had the user enter the text siring 586SX into tlie 
user information 74 when the user was purchasing software for example, when 
the user hit enter, the user applet 30 would be informed that one of the objecis , 
5 of the form in its array had been changed, particularly that the array displax ing 
the user information would now contain a text sirmg. 1 he user applet 30 
would communicate this to the server 20 which would then lei all the applets in 
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the session, in this case the service applet 22. I he service applet 22 would then 
evaluate each object in the array and ascertain which object had changed. After 
identifying that the user input 74 had changed to now include a le.xt string 
.S86SX it would update the shared content. 

Figure 4A .shows the user view used in the previous example, vvliere the 
user view 00 having the .shared view 92. The shared view 92 is es.sentially a 
child browser spawned by the user applet 22 to display I ITMI, files, f hc 
shared view 92 in this example has the URI. 96. the text 98. the user 
information 100 and the text .string 586SX 102. In this manner, cither the user 
computer 12 or the second computer 24 can interactively enter information into 
the form that will be displayed to the other party in a real time as previously 
described. Thus, instead of having the second computer 24 merely push the 
appropriate form to the user computer 12, the user could complete the form 
while in the session and the sales representative could audibly communicate to 
the user whether the user was properly or improperly completing the lorni. 



In the previous example, the applet was informed that there was an update 
oi the object by either the user computer 12 or the second cemipuler 24 hittini: 
enter. This i.s not necessary for other entries on the form such as a check the 
box entry. For such entries, the user applet 22 could simply have two values. 

20 true and false represented by zeros and ones. Each time a check was formed in 
the box. the user applet 22 could inform the server 20 of the change in the 
object. The server 20 would then inform all of the applets in the session that 
there has been a change and appropriately update the representation of the 
shared content to include the check. As such, any of the forms on the Internet 

25 are capable of being completed in this manner. 
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In another embodiment, the form could also be a two dimensional array 
where in each frame of a form is the first argument, and the second argument is 
the objects on each form. In this embodimenl, the first frame would be 
designated Irame one, the second would be designated as frame two and the 
5 third designated as frame three. 

In still another embodiment, the form could contain minor modifications 
that allow it to notify the service applet 30 or the user applet 22 when a change 
occurs in a field, similar to event monitoring. In still a further embodiment, 
applets can be generated to represent the objects of the f orm. In this 
K) embodiment the applets representative of the objects would notify the user 
applet 22 or the service applet 30. as the case may be of changes 10 the form. 
In the preferred implementation, the applets would be Java applets. 

If the user computer 12 is attached to a server that disallows an outside 
server 20 to read information from a form, the user computer 12 can enable 
15 services such as data tainting in Netscape Navigator or other well known 

methods to enable the server to see the changes in the form. In data tainting, 
the user computer 12 explicitly gives permission to read the information from 
the document that the computer is viewing. 

In addition to the shared view 92. the user interface 90 also has a control 
20 view 94. which is framed adjacent to the shared view 92. The control view 94 
allows the user computer 12 to manipulate the session and can include features 
such as, for example, send 104, a box to type an URL location, as well as 
disconnect. The send 104 on the control view 94 can be selectively disabled by 
the second computer 24. to allow the second computer 24 to control the 
25 interactive visual session when a user computer 12 is manipulating the shared 
content improperly. 
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20 



Turning now lo Figure 4B, where like numerals dcnoic previously 
described elements, there is also shown a second embodjment of the user 
niterlace 90 •. The user interface 90' that contains a control view 94" and a 
shared view 92\ however, in this embodiment, the shared view overlies the 
control view and is not framed within the browser 18 of the user computer 1 2. 
This is the preferred embtidimcnt as web pages are more commonly displayed 
as one page m a browser and not as frames so that each page can be 
individually book marked and print as users are accustomed to. Further, as the 
user computer 12 docs not have to sec in the comrol view it is not necessary to 
frame the web pages shown in I-igure 4A. In particular, as there is no queue 
information in user view regarding its queue once a session has established, the 
u.ser computer does not need to have a framed page as shown in I- igure 3. 

The shared view 92' of Figure 413. displays shared content ol an 
application wherein the application displayed is a spread-sheet 120 having 
menu controls 122. To share an application, one of the computers in the 
session must have the application running on its computer, hi the preferred 
embodiment the application is running on the .second computer 24. If the user 
computer 1 2 is the one who wanted to share the page, they could select sharin- 
the excel page by activating a button or press a hard key. for example, that 
states "Send to Webline '. When either party hits the send button, an 
implemeniaiion grabs the screen as an miage and then does an File Transfer 
Protocol C FTP") or Hyper-text Transfer Protocol (^HTTP") to put ii on a dn^e 
of the server 20. It should be obvious that there are other ways to implement 
application sharing that may be mcoi-porated into the invention without 
deviating from the scope of the invention. In another embodiment, the shared 
application could be running on the server 20 or any server connected to the 



wo 97/42728 . 33 . PCT/US97/07740 



-network 16. with the second computer 24 and the user computer 1 2 both 
sharing the same view of the application. 

In the preferred embodiment a plug in for Netscape Navigator or an aciive 
X-control adapted for a Microsoft IE browser is used. In this embodiment. 
5 when the user hits the send button the Active X control grabs the screen image 
using Windows API functions or similar functions well known in the art. 
However, once the Java server receives the image, it notifies the applets in the 
session that it has received the image. The current image to be shared is given 
a random and unique URL location which is then sent to the other computer's 
10 applet across the network 16 just as if they were sharing a URL of a web page. 
The process is repeated for each image of the application that the panics want 
to share. 

Image sharing can also be implemented by just transmitted the changes 
between one image and the next, and thus reserve resources on the server. 
i5 liither way. the user computer 12 and the second computer 24 can share an 
application without requiring the u.ser computer to be actively running the 
application. 

In a funher embodiment, any activity (e.g. key clicks, mouse clicks 
etc.) on the shared view of the application can be transmitted to the computer 
20 running the application and any resulting new changes can be shared. 

Turning now to Figure 5. where like numerals denote previously 
described elements, which .shows one embodiment of the present invention 
wherein more than one second computer 24. 1 50 is connected 10 a L.AN 1 52 
which is then connected to the server 20. Computer 150 can be identical to the 
2.^ second computer 24 having an identical service applet 30 or it can be different. 
It can contain an administrative applet, for example. 



BNSDOCID: <WO 9742728A2_L: 



wo 97/42728 



- 36 - 



PCT/US97/07740 



I his system allows the second computer 24 to have computer 1 50 . join a 
session upon request by placing in a queue display 1 58 of computer 150 a 
request to join tiic session. When computer 150 activates the connect 160 
feature in the queue display 158, the server 20 through the LAN 152 would 
5 download to the computer 1 50 the relevant session information as well as an 
applet, if necessary. It should be obvious to those skilled in the art that the 
second computer 24 can also transfer the session to computer 1 50 and not just 
add computer 1 50 to the session. 

The server 20 is again connected to the network 16 for communication 
t thereon, and the user computer 1 2 is connected to the network 1 6. 

A phone 164 of the user is explicitly show connected to a public switched 
telephone network ("PSTN") 166. As such, the user 162 con.stitutes both the 
phone 64 and user computer 12. I he second computer 24 also has a direct 
connection to the PSTN 166 either through a local exchange or a long distance 
service. As such, when the second computer 24 is informed by the server 20 
that a call request from u.ser 162 is tn its queue, the second computer can 
connect through the public telephone network to the local PSTN 166 ot the 
user 162 and to user's phone 164. 

In this embodiment, the server has a database 154 disposed thereon. As 
previously described, the database .stores information regarding the queue, 
assignments, attributes, and password validations among other items. 

Turning now to Figure 6, where like numerals denote previously described 
elements, which shows another embodiment of the present invention wherein 
more than one user 162. 164 is shown. User 162 has the user computer 12 and 
the phone 164 as was previously described. User 164 can also have a user 
computer 166 and a phone 168. Wherein both user computers 12. 106 are in 
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communicaiion with network 16 and both user phones 164. 168 are in 
communication with a PSTN 166. The second computer 24 can enter a call 
session with user computer 166 as well as enable the user computer 12 to enter 
the same call session. 

5 In this embodiment, a sales center 1 88 is shown comprising the 

salesperson 1 90 having the second computer 24 and a phone 180. The phone 
180, however, is connected to an automated call distribution system (^'ACD'") 
1 82. ACDs arc well known in the art, and they have, among other features, 
more than one queue 188 disposed thereon within which to route calls Irom ihc 
10 publicly swiiehed network or data networks. In this embodiment, when u.ser 
162 enters a call request through the phone 164, the call is transmitted to the 
PSTN 1 66 and then to the ACD 1 84 which places it in a queue 1 88. Vhc ACD 
could thereafter place the telephone call for the sales representative 190. 

However, the ACD is also connected via 1 86 to the network I 6. This 
I .^ enables either the queue information 1 88 of the ACD to be communicated to 
the server 20 and displayed by the server 20 to the more than one second 
computers in the sales center 1 88 and the user computer 12. 166. 
Alternatively, the call requests from the network 1 6 that go to the server 20 be 
communicated to the ACD. Having the ACD connected to the Internet in this 
20 manner, allows the system to use the sophisticated aspects of an automated 
ACDs. such as queue service, to supplemeni or substitute for the computer 
queue functions or other similar functions in the server 20. 

In this embodiment, a firewall 192 is also disposed between the user 
computer 12 and the server 20. The firewall 192 can be configured so as to slop 
25 all incoming traffic from the network 16. stop all outgoing traffic from the user 
computer 12 or any variation there between. Typically, the firewall 192 will 
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allow ccnain information to pass, but restrict the use of that information. For 
example, the firewall will allow applications in from the network 16. but once 
disposed the user computer 1 2 the application downloaded from the network 
1 6 will not be allowed to function upon documents disposed on the user 
5 computer 12 thai are not generated from the same server as was the application. 
Such a restriction disables the user applet 22 from modifying interactive forms 
within user computer 12. 

The user computer 12 can expressly permit the function of the user applet 
22 and the server 20 in each individual instance. In the preferred embodiment. 

]U however, the user applet 22 functions are achieved by lalkm^ through the 
firewall at port 80 using HTTP protocof As it should be obvious lo those 
skilled in the art, this port is usually left open by a firewall for communication 
over the worldwide web. It should be also obvious to those skilled in the art 
that other ports can be used with other modifications or any iniermediar\ 

15 solution can be found wherein the user computer 12 grants permissions to the 
user applet 22 to perform functions. 

In the preferred embodiment, however, the user computer 12 as is typical 
is allowed to connect to an external port of a random server as long as it is 
using HTTP protocol and as long as Ihey arc conncciing to port 80 of another 

20 server. Using these specifications, the system performs HTTP tunneling. 

Accordingly, in the preferred embodiment the user applet 22 is sent from port 
80 communicating in HTTP protocol lo look very similar to a Common 
Gateway Interface (^X:GI") request, and is allowed in by the firewall. 
Thereafter, communication from the user applet 22 still has to be in a protocol 

25 capable of going through the firewall. As such, any request from the user 

applet 22 in this embodiment looks like a standard HTTP request from a web 
page and as such looks like a CGI request. The firewall will let the request 
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through from the user applet 22 onto the network 16. The request is in the 
form of a function caller request, which is transmitted as a message to a CGI 
handler. The CGI handler translates it, passes it to the CGI programmer which 
translates the CGI request to the server 20, which in the preferred embodimeni 
5 is a .lava server. Any transmission back from the Java server 20 to the user 
applet 22 is done through the same mechanism. Hssenlially, the 
communication between the server 20 and the user applet 22 is cloaked to look 
as is it is a request from a web site on the Internet and is such is allowed to go 
through the firewall 192. 

10 It should be obvious to those skilled in the art that using either a dilTereni 

server or different handlers or other known methods to go through the firewall, 
including any port or other configuration, is possible without departing from 
the scope of this invention. 

l urning now to Figure 7A. which shows one method by which the user 
]5 computer 12 can enter a call session with the second computer 24 and thercbx 
providing audio communication across the network 16 or the telephone 
network in conjunction with visual communication across the network 16. As 
is previously described, the user computer 12 can enter into a call session with 
the second computer 24 by numerous methods. 

20 In Figure 7A the request to enter a call session starts 200 when a user of 

the user computer 12 is instructed by the sales representative of the second 
computer 24 via telephone to go to a specific web page 202. Displayed on that 
web page will be a sign-on page wherein the user is prompted to enter 
attributes, such as the users name or any other attributes previously described. 

25 In this embodimeni. as the user and the sales party are already in audio 

communication, the user will already have been provided by the sales 
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reprcsenlaiivc a password thai uniquely identifies to the server 20 the session 
the user computer 12 is supposed to enter. As such, the user computer 12 is 
also prompted lo insert a password 206. After the user enters the password, the 
server 20 places the call request in computer queue 208 as was previously 
5 described. 

At this point the server 20 also transmits to the user computer 1 2 a user 
applet 22. The user applet 22 generates a child browser in the user view of the 
user computer 12 and then begins to poll the server 20 for call request status 
212. If the call request status is noi simultaneously activated bv the service 
n» applei 30, then the user computer 12 is displayed the on-hold information 2 1 6. 
The on-hold information will include, for example, the user computer* s status 
in the computer queue, any advertisements as previously described, and other 
additional inlbrmation as required. 

fhe control view can also be selectively displayed by the user computer 
15 218. However, in the preferred embodiment as the control view is not framed 
within the user interface. As such, the control view will be sitting under the 
child browser, and the child browser will display the on-hold information. 

1 he user applet 22 will continue to poll the server 20 for the call request 
status 220, If the caller request status is that the user computer is still in the 

20 computer queue, the user applet 22 will update the computer queue information 
still in the computer queue 222, then the user applet 22 will collect the updated 
computer queue information 224 and redisplay the on-hold information in the 
child browser 216. If. however, the second computer 24 has selected the call 
request of the user computer 12, the service applet will notify the server 20 that 

25 the user computer having that session identifier can now be entered into a call 
session 224. After the panics are entered into the call session the 
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communication beiwcen ihem and the manipulation of the shared consiani on 
the user interf ace of the user computer 12 and the user interface of the second 
computer 24 will continue as previously described. 

Turning now to Figure 7B, which shows a second method by which the 
5 user computer 12 can enter a call session and start 200 the visual interaction 

between the user computer 12 and the second computer 24. In this method it is 
assumed that the user and the sales representative are not initially in audio 
communication. Instead, it is assumed that the user is on the worldwide web 
and is viewing the web page 228 of the sales organization. 

10 11 the user of the user computer 12 wishes to enter the session the user can 

click the call request button 230 on the web page of the sales organization. 
Upon clicking the call request button 230, the user computer 12 displays on its 
browser a form prompting the user to insert attributes 232 such as name, 
interest, and so on. After the user computer enters the attributes 232 the 

15 attributes are communicated through the network 16 to the server 20. 

whereupon the server 20 passes the attributes 232 via methods commonly 
known in the art. The server 20 then goes through skills based routing 234 as is 
previously described and determines which computer queue is appropriate for 
the call request. 

20 The server 20 then places the call request in the computer queue 236 The 

call request in the computer queue can be communicated to the ACT) or 
displayed on the computer queue of the second computer and the user 
computer. The ACD can also deiermmc which queue is appropriate for the call 
request in place of the server and transmit the corresponding selection of the 

25 computer queue to the server. 
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Regardless, the server then transmits the user applet 238 to the user 
computer. The user applet spawns the child browser and displays the on-hold 
information 240 to the user. Again, the browser of the user computer ] 2 can 
selectively display the control view 242 in a framed method or behind the child 
5 browser. Similarly, the on-hold information can include not only information 
regarding the call request status in the queue but also the advertisements 
determined either based on the attributes or jusl the advertisements for the sales 
representatives' company in general, as is previously described, hi one 
implementation if the ACD will place the calL the queue of the ACD would be 
M; displayed as communicated to the server, while in another implementation, the 
computer queue of the scrx'er would be displayed, it should be obvious that ihc 
queue of the ACD and the computer queue of the server can supplant each 
other or work in conjunction with each other as is nccessar\ . 

While the browser of the user computer 12 is displaying the on-hold 
I information, the user applet 22 continues to poll ihe server 20 for the call 

request status 244. If the user applet 22 is inibrmcd by the server 20 thai the 
call request is still on the queue 246 then the user applet 22 will collect the 
updated on-hold information 248 and redisplay the on-hold information m the 
child browser 240. This process will continue until the user applet 22 polls the 
20 server 20 for the call request status 244 and is told that the service applet has 
enabled the call request of the customer and informed the server 20 of the 
same. Alternatively, the ACD can place the call and inform the server that the 
call request has been enabled. At this lime, the user applet receives notification 
that the user can enter the call session 250. 

25 The invention may be embodied in other specific forms without depaning 

from the spirit or essential characteristics thereof The pjesent embodiments 
ure. therefore, to be considered in all respects as illu.strative and not restriciixc. 
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the scope of the invention being indicated by the appended claims rather than 
by the foregoing description, and all changes which come within the meaning 
and range of equivalency of the claims arc therefore intended to be embraced 
therein. 
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Claims 

The embodimenls of the invention in which an exclusive property or 
privilege is claimed arc defined as follow^s: 
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1 . A method of providing visual interaction across a network between at least one 
user computer having a browser for reading files and at least one second 
computer in communication with a server, the server interposed between the at 
least one user computer and the at least one second computer, the method 
5 comprising the steps of: 

establishing a link between the at least one user computer and the server; 

selectively transmitting a user applet to the at least one user computer, the user 
applet enabling the at least one user computer to communicate with the 
server; 

10 generating a user view on the at least one user computer; 

communicating that the link is established to a service applet disposed in the al 
least one second computer, the service applet adapted lo enable the at 
least one second computer to communicate with the server; 

establishing a session between the at least one user computer and the at least 
1 5 one second computer; 

enabling the at least one second computer to display a service view containing 
a representation of the user view; 

selectively manipulating the representation of the user view by the at least one 
second computer; 

20 communicating to the service applet changes in the representation of the user 

view; 
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transmitting the changes in the representation of the user view from the service 
applet to the user applet; and 

implementing any changes communicated to the user applet in the user view, 
thereby altering the user view to confomi to the representation of the 
user view of the at least one second computer, thus enabling the at least 
one second computer to communicate to the at least one user computer. 
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A method according to claim 1 further comprising the steps of: 

selectively manipulating the user view by the at least one user computer; 

communicating to the user applet change in the user view; and 

transmitting the change from the user applet to the service applet, thereby 
3 allowing the at least one user computer to control communication with 

the at least one second computer. 
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3. A method according to claim 2 wherein the ai least one second computer can 
disable an ability of the at least one user computer to selectively manipulate 
the user view. 

4. A method according to claim 1 wherein the user applet and the service applet 
bi-directionally communicate. 

5. A method according to claim 1 wherein each of the at least one user computer 
is adapted to make call requests to the server, thus requesting voice 
communication. 

6. A method according to claim 5 further comprising an automated call 
distributor in communication with the server having at least one queue adapted 
to receive the call requests from the server. 

7. A method according to claim 6 further comprising at least one computer queue 
disposed on the server adapted to receive notification from the automated call 
distributor of call requests and update the at least one computer queue. 

8. A method according to claim 5 wherein the server further comprises at least 
one computer queue disposed on the server adapted to seleciivelv receive the 
call requests routed by server. 

9. A method according to claim 5 further comprising: 

at least one computer queue disposed on the server adapted to receive the call 
requests selectively routed to the at least one computer queue by the 
server; and 

an audio module disposed on the at least one second computer adapted to 

enable each of the at least one second computers to access the at least 
one user computer having the call requests in the at least one queue. 

10. A method according to claim 1 further comprising: 
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at least one computer queue disposed on the server having the call requests; 
and 



queue information representative of the call requests disposed in the al least 
one computer queue, the queue information displayed on the at least 
one second computer. 
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11. A method according to claim 1 0 wherein the server updates the queue 

information that is displayed on the ai least second computer in real time. 



A method according to claim 10 wherein the server transmits to the at least 
one user computer supplemental information while the call requests of the at 
least one user computer is in the at least one computer queue. 

13, A method according to claim 12 wherein the supplemental information is 
selected according to attributes dictated by the at least one user computer. 

14. A method according to claim 1 further comprising the steps of: 

extracting from the at least one user computer user attributes representative of 
the at least one user computer; 

ascertaining service attributes of the at least one secured computers; and 

selectively communicating that the link is established to selected ones of the at 
least one second computers by comparing the user attributes and the 
service attributes. 
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15. A method according to claim 1 further comprising the step of selectively 
adding an additional computer to the session by the server. 

16. A method according to claim 1 5 wherein additional computer is disposed 
remotely from the at least one user computer and the at least one second 

5 computer. 

1 7. A method according to claim 1 wherein the session is password protected. 

18. A method according to claim 1 wherein the service view further comprises a 
script frame adjacent to the representation of the user view and having a 
representation of one or more scripts disposed therein, each of the one or more 
scripts being adapted to communicate with the service applet and being 

5 adapted to control the representation of the user view. 
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A method according to claim 18 wherein the script initiates a file transfer to 
download information to the user view from a remote location. 

A method according to claim 1 8 wherein the script is dynamically defined 
according to attributes of the at least one user computer. 

A method according to claim 20 wherein the script is dynamically defined 
according to attributes of the at least one second computer. 

A method according to claim 20 wherein the script is dynamically defined by 
the second computer to generate a comparison of information within a display. 

A method according to claim 22 wherein the script is representative of a short- 
cut to enable the at least one second computer to push predetermined 
information to the at least one user computer. 

A method according to claim 1 wherein the any changes to the representation 
of the user view is dictated by a change of a address of the user view, the 
service applet communicating the change of address to the user applet. 

A method according to claim 1 wherein the changes to the representation of 
the user view is forced by the service applet which communicates an image to 
the user applet. 

A method according to claim 25 wherein the image is transmitted to the at 

least one user computer at predetermined time intervals. 

A method according to claim 25 wherein the captured image is transmitted to 
the at least one user computer in real time. 

A method according to claim 25 further comprising the steps of: 

assigning an image address to the image, the image and the image address 
disposed within the server; 
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communicating lo the user applet the image address; and 

displaying the image in the user view of the at least one user computer and the 
representation of the user view in the at least one second computer. 
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29. A method according to claim 1 wherein the changes to the representation of 
the user view is captured as image changes, where the image changes are 
representative of a portion of the image that has been selectively manipulated, 
the service applet communicating the image changes to the user applet. 
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30. A method according to claim 1 wherein the user view and the representation of 
the user view is a form having objects, the method further comprising the steps 

of: 

selectively manipulating the objects by the at least one user computer; 

selectively manipulating the objects by the at least one second computer; 

transmitting the objects to the service applet and the user applet; and 

displaying the objects in the representation of the user view and the user view 
thus allowing the user view and the service view to display the. form as 
the form is completed. 

31. A method according to claim 30 wherein the step of transmitting the objects 
further comprises the steps of: 

querying the form to ascertain the objects disposed on the form; 

representing the form as an array having arguments representative of the 
objects; 

selectively updating the objects such that the arguments of the array represent 
any changes to the objects; and 

transmitting at least portions of the array to the service applet and the user 
applet. 
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A method according lo claim 31 wherein the at least ponions of the array are 
representative of only the any changes to the objects. 

33. A method according lo claim 30 wherein the form is enhanced by making the 
objects of the form event driven, wherein the step of transmitting the objects 
further comprises the objects communicating to the applet when the objects 
arc selectively manipulated on the form. 

34. A method according to claim 30 wherein form is enhanced by generating Java 
applets representative of the objects wherein the step of transmitting the 
objects further comprises transmitting from the Java applets any changes to the 
objects to the user applet and the service applet. 

35. A method according to claim 1 wherein the user view is a software application 
disposed on the server. 

36. A method according to claim 1 wherein the user view is a software application 
disposed on the at least one second computer. 

37. A method according to claim 1 wherein the user view is a software application 
disposed on the at least one user computer, 

38. A method according to claim 1 wherein the user applet generates the user 
view, the user view having a child browser. 

39. A method according to claim 1 further comprising the step of validating the at 
least one user computer by the server prior to establishing a session. 

40. A method according to claim 1 further comprising a database adapted to store 
queue and session information. 

41 . A method according to claim 1 wherein the server and user applet selectively 
communicate through a predefined port thus enabling communication through 
a firewall. 
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A method according to claim 1 wherein the at least one user computer requires 
no additional pre-installed executablcs to communicate with the at least one 
second computer. 

A method according to claim 1 wherein the at least one user computer and the 
at least one second computer can disconnect themselves from the session. 

A method according to claim 1 wherein the at least one second computer can 
selectively disconnect another computer from the session. 

A system for enabling communication between a service party having at least 
one second computer and a user of a first computer, the system comprising: 

a server; 

a user view displayed on the first computer; 

a user applet downloaded to the first computer by the server and in 

communication with the server, the user applet in communication with 
the ser\'er and adapted to generate the user view; 

a service view disposed on the at least one second computer, the service view 
having the user view displayed therein; and 

at least one service applet disposed within the at least one second computer in 
communication with the server, the service applet having of 
functionality required by the at least one second computer to 
manipulate the service view, whereby the service party is adapted to 
communicate to the user by manipulating the user view through the 
service view. 

A method for enhancing a resource collection function of a computer in 
communication with a network while enabling the computer to use 
functionality of a resource, the method comprising the steps of: 
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defining functions of the resource thai the user wishes to enable, the functions 
- of the resource being a sub-set of the functionality of the resource; 

generating an applet representative the functions of the resource: and 

communicating the applet to the user\s computer as the resource collection 
function, thereby decreasing the resource collection function of the 
user's computer while enabling the functions of the resource the user 
10 wishes to enable. 

47. A method of controlling a browser view on a first computer from a script 

displayed separately from the browser view, the script communicating with an 
applet, the applet adapted to generate the browser view, the method 
comprising the steps of: 

providing addresses as simplified commands in the script; 

defining the applet to receive the simplified commands as arguments; 

selectively activating portions of the script; 

associating the simplified commands of the portions of the script with the 

applet such that the applet will generate the browser view dictated by 
*0 the portions of the script that are activated; and 

modifying the browser view in accordance with the simplified commands. 
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48. A method according to claim 47 further comprising a remote computer having 
a user view and a user applet in communication with the applet, the user applet 
adapted to receive the simplified commands from the applet and modify the 
user view of the remote computer. 

49. A method according to claim 48 wherein the applet has session information 
representative of the first computer and the remote computer. 
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•SO- 
SO. A method according to claim 48 wherein the step of providing addresses as 

simplified commands in the script further comprises the steps of: 

ascertaining user attributes of the remote computer; and 

selectively generating addresses such that the addresses relate to the user 

attributes thus allowing the browser view to contain a display related to 
the user attributes. 
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51 . A method according lo claim 50 wherein the server selectively generates the 
addresses. 
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52. A method according to claim 50 wherein Ihc first computer selectively 
generates the addresses. 
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- 63 - 

53. A method according lo claim 48 wherem tht step of providing addresses as 
simplified commands in the script further comprises the steps of : 

ascertaining session attributes representative of service attributes of the second 
computer and user attributes of the remote computer; and 

selectively generating addresses such that the addresses relate to the sessi(^n 
attributes. 
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